Fuzzy inference system or adaptive neuro-fuzzy inference system, and intelligent agent for the dynamic generation and retrieval of user interface software modules

ABSTRACT

A method of operating a target system wherein small changes in input variables produce small changes in output variables in a manner permits system learning on a time dependent basis. The rate of change of the system output is directly dependent upon the product of the rate of change of the system input and a matrix consisting of input variables and antecedent and consequent parameters. A preferred performance criterion is obtained through the approximation of the said matrix to a weighted-augmented pseudo-inverse Jacobian. Off-line, the system undergoes a series of iterations using a wide range of input variables wherein the actual outputs are compared to the desired outputs and optimized values for the antecedent and consequent parameters are obtained and passed back to the said matrix for use in the subsequent iteration.

BACKGROUND OF THE INVENTION

Fuzzy Inference Systems (FIS) and Adaptive NeuroFuzzy Inference Systems(ANFIS) exist for use in the operation of a target system. FISs andANFISS are used in control problems and decision making problems. Withcontrol problems, the FIS or ANFIS act as a feedback control for thetarget system so that the inputs to the target system are properlychanged based on actual system outputs. With decision making problems,the decision making rules of the target system may be “fuzzified” toprovide for better decision making.

While FISs and ANFISs have been applied to many target applications, ithas not been contemplated to apply an FIS or an ANFIS to the control ofa user interface such as a screen display.

Over the past decade, many software programs have been developed for thedisplay of descriptive information such as: multi-media units, webpages, graphic user interfaces (GUIs), cd-rom information, graphics,diagrams, etc. Most of these software programs are conceived to generateand display the information in a static fashion. That is, once an outputis generated, the user (although he/she can manipulate the displayedinformation in a given framework) does not have a priori the means tochange the structure and preferred way to display the availableinformation. Consequently, and this is more true for multi-media andweb-pages modules, the retrieved module may contain unnecessary (from aparticular user point of view) information that makes its displaycumbersome and/or lengthly and, consequently, time consuming.

Recently some commercial packages are starting to appear that provide aparticular user some a priori options to display (web-pages, multi-mediaunits) in an active/dynamic fashion the available information. However,the a priori options are few and rather limited. That is, once an optionis selected the display module is generated according to some simplerules. These rules do not take into consideration a wide range ofexperts knowledge, nor do they display the information on the principlethat for some small changes in the input, there should be some smallchanges in the output. Furthermore, it is very important to notice thatthe options to the user are currently of a crisp nature.

SUMMARY OF THE INVENTION

This invention seeks to develop an advanced FIS or ANFIS as anIntelligent Inference System which is useful for a wide variety ofapplications including the control of a screen display.

According to the present invention, there is provided

The development and application of an Intelligent Agent for the dynamicgeneration/retrieval and operation of descriptive/display softwaremodules.

The development and application of an Intelligent Inference System fordecision making/resolution cases: in particular for the design ofdescriptive/display software modules.

The development and implementation of an Adaptive Neuro-Fuzzy InferenceSystem as an Intelligent Inference System.

The development and implementation of an Adaptive Fuzzy Inference Systemas an Intelligent Inference System.

The development and implementation of a Fuzzy Inference System as anIntelligent Inference System.

The development and implementation of a Performance Criterion for theIntelligent Inference System.

An active and dynamic input interface that allows crisp and fuzzyinputs.

A crisp and fuzzy (input) icon.

BRIEF DESCRIPTION OF THE DRAWINGS

In the drawings which describe example embodiments of the invention.

FIG. 1 is a software block diagram of an intelligent agent made inaccordance with this invention.

FIG. 2 is a perspective view of a computer which may use the intelligentagent of FIG. 1.

FIG. 3 is a block diagram illustrating an FIS or ANFIS made inaccordance with this invention together with a target system. and

FIGS. 4a, 4 b, 4 c, 4 d and 4 e comprise a flow diagram for softwarecontrol for,use in developing an FIS or ANFIS made in accordance withthis invention.

FIGS. 5a, 5 b, 5 c and 5 d are schematic variants of icons which may beused with this invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The screen display control system developed here is envisioned as anintelligent, generic, interactive, user friendly unit, and computerplatform independent: with the purpose of properly generating anddisplaying a module in a way preferred/selected a priori by the user.

Here, “intelligent” has the connotation of incorporating a wide range ofExperts knowledge in the design rules: and then modifying/optimizing it(based on a novel Performance Criterion) according to the principle thatfor small changes in inputs it should correspond a small change in theoutputs.

Furthermore, it is also important to mention that the Intelligent Agentis devised to deal with user options of a crisp and fuzzy nature.

Two types of users are foreseen:

Type I. Active procreators, “authors”.

Type II. Active recipients, “readers”.

Here “active” has the connotation of dynamically interactive andcontrary to a possible passive static fashion as previously mentioned.On the other hand, procreator has the implication of being the originalmodule creator.

The Intelligent Agent is devised such that a human being (Type I) caneasily interact with a user friendly active interface by answeringquestions (of a crisp and/or fuzzy nature), and/or selecting options viaa crisp/fuzzy icon. The software system will then instantaneouslygenerate a dynamic descriptive module. This module can beinstantaneously accessed by the Type I user and be retrieved to bedisplayed as preferred at any time: and/or also be instantaneouslyretrieved and displayed as preferred by Type II users in other (whichcan be distant) server sites at any time.

The Intelligent Agent (10) with reference to FIG. 1, is an integratedsoftware system consisting of a Coordinating Module (12) which properlycoordinates and manipulates inputs/output of an Active Input Interface(14), an Intelligent Inference System (16), and an Output Interface(18). The Intelligent Inference System is the core of the IntelligentAgent. It consists of an Expert System (20), a Fuzzy Logic InferenceSystem (22), and an Adaptive Neural Network (24). Here, some Schemes forproper parameter scaling, tuning/updating, and learning are provided,such that the Intelligent Inference System can be constructed andoperated as: a Fuzzy Inference System, or as an Adaptive Fuzzy InferenceSystem, or as an Adaptive Neuro-Fuzzy Inference System.

Unlike other current intelligent inference systems, here the devisedsystem is conceptually considered, in a novel fashion, at the rate ofchange level. Under this proposed framework a novel criterion (i.e.measure) of system performance is derived. This criterion facilitatesthe implementation of a type of control system to the decisionanalysis/resolution for the module design. Furthermore, this criterionalso permits the proper tuning and updating of the antecedent/premise,scaling, and consequent parameters in the Fuzzy Logic Inference System.As a result the devised Intelligent Inference System exhibits somedesirables features and properties (such as efficiency, reducedsubjectivity, stability with respect to small changes in the systeminputs) missing in other current intelligent inference systems.

The gathered information via the Active Input Interface is passed to theCoordinating Module. Then, it is properly transferred to the IntelligentInference System which in turn generates a decision resolution(diagnostic) for the proper design of the descriptive module. Thisdecision resolution has taken into account user options and the expertsknowledge: it can be considered as an intelligent compromise. The resultof the decision resolution is finally feed into tie Output Interface forthe actual generation and display.

The Coordinating Module of tie Intelligent Agent is a software programwith the purpose of manipulating and coordinating the inputs and outputsof the Active Interface, the Intelligent Inference System, and theOutput Interface.

Since the Intelligent Agent system is envisioned as an intelligent,generic, interactive, user friendly unit, and computer platformindependent: it is assumed here that the system installation aspects andinitialization protocol are easily accomplished. It is also assumed thatthe Type I user for the generation mode has prepared and installed orassembled some descriptive sub-units (such as text, audio, film, video,graphics, diagrams, link files, etc.) in a specified yet general(presumably, but not necessarily, hierarchical, prioritized order)format. It is also assumed that the Type II user has installed the sameIntelligent Agent system.

In the generation mode, the preferred options are requested via theActive Input Interface, and then from the selected ones the properinformation inputs are feed into the Intelligent Inference System. Theoutput is then used to properly set the command tags/labels (values) ina generated (advanced Hypertext Language type such as, but notnecessarily, the HTML: or the Extensible Markup Language XML)descriptive module for screen display. In the retrieval mode, similarsteps are taken. However, in this case there is no actual modulegeneration: rather, the target module is the retrieved descriptivemodule for screen display to which the updated tags/labels (values) arejust passed.

The tags are generally filename extensions. By setting the values ofthese tags, selected files (module sub-units) can be activated orignored. In this way, text in a file, images/pictures, static/dynamicgraphics, figures, frames, icons, etc, would be properly placed or wouldnot be displayed: or an audio or video file (module sub-units) would beplayed in a proper order, or would not be played.

The Active Input Interface is an input interface which consists of asoftware program that generates (of a crisp and fuzzy nature) questionsand/or display possible options, via a crisp/fuzzy icon, for Type I orType II users: and then accepts the answers and/or preferred options.

For the generation mode the posed options (questions) to select can berelated, amongst others, to the following subjects:

Nature of module

The content of the module;

The purpose of module.

Display Information attributes:

Spatial organization (placing of text, graphics, etc.);

Structural hierarchy (division of chapters, sections)

Levels

Design Features

The preferred generation/retrieval format (frames, multiple layers,etc.)

Decorative features (icons, marks signs, etc.)

Special features (such as links to other sites, link maps, etc.)

For the retrieval mode, one could additionally or instead select somedesired features related to:

Retrieval priorities;

Contents percentage:

Module Length:

Browse features.

Although the user should have as many options as possible, the availableoptions should be designed to facilitate interaction and comply with asimple and “intelligent” mode of operation as much as possible.

The collected information then is passed to the Coordination Softwaremodule which manipulates it to properly feed it to the IntelligentInference System, and to the Output Interface. There are several ways toconceive a crisp/fuzzy icon. One that permits easy interaction and doesnot require much familiarization is the following. The icon of FIG. 5ais displayed, as current conventional ones, at the click of a crispbutton. But each layer of displayed option is designed such that each ofits slots is divided in two horizontal parts. The lower (or conversely,in FIG. 5b, the top) part 50 is a fraction of the size (i.e. 0.2) of theupper (or bottom) one 52. The upper (or conversely, the bottom) one isfurther divided vertically in several parts 54, 56, 58: eachcorresponding to a range of the fuzzy attribute of the selected option.The lower (or conversely, in FIG. 5b, the top) part 50 in the other handis just a crawl bar that depending on where is clicked, it transmits acorresponding attribute value. In this manner the user has the option totransmit preset ranges of values of fuzzy attributes: or a specifiedvalue for a fuzzy attribute. In the case of a crisp option, thecorresponding slot is just a conventional one.

A variant to this model, seen in FIG. 5c, consists on dividing thedisplayed option in two vertical parts. The left (or conversely, in FIG.5d, the right) part is a fraction of the size of the right (or left)one. The right (or conversely the left) is further divided horizontallyin several parts: each corresponding to a range of the fuzzy attributeof the selected option. The left (or conversely, in FIG. 5d, the right)is just a crawl bar that depending on where is clicked, it transmits acorresponding attribute value.

First, it should be mentioned that there are many ways to design, built,and implement intelligent inference systems. Also, there are manytechnical issues and aspects that can influence/affect their efficiency.Usually, intelligent inference systems are developed for the intelligentcontrol of systems. Furthermore, they are usually formulated at thefunctional level.

Here, a proper framework to develop intelligent inference systems isconsidered. That is, here the developed intelligent system is gearedtowards intelligent decision analysis and resolution, and applied to thedesign of descriptive/display modules. Unlike current intelligentinference systems, here the devised system is conceptually considered,in a novel manner, at the rate of change level.

Under the proposed framework a relationship between the rate of changeof the outputs with respect to the rate of change of the inputs isestablished. From this relationship it is quite straight forward toidentify a characteristic matrix, from which a novel criterion of systemperformance can be derived.

The presented framework has been devised to address and deal effectivelywith the following issues. In most decision making and resolutionproblems it is difficult to identify a priori a reference, to justify ordefine a proper target error. Therefore, for this kind of problem it isdifficult to implement intelligent inference systems which wereoriginally conceived for systems control. Also, it is difficult toconceptually ensure that open-loop intelligent control systems willyield small outputs for relatively small inputs, if applied to decisionanalysis problems of a highly non-linear nature. The devised formulationpresents the following desirable characteristics:

Reduce any possible subjectivity introduced in the Expert system:

Establishes a proper criterion to deal effectively with the inherentdecision analysis and resolution prproblem for the module design.

Facilitates dealing with decision analysis and resolutions issues:

Ensures that for small changes in the input, small changes in the outputresult;

Permits a fine tuning and updating of the membership functions in theFuzzy Inference System.

The devised Intelligent Inference System is envisioned to include anExpert System, a Fuzzy Inference System. and an Adaptive Neural Network.Once the parameters of the Fuzzy Inference System (FIS) are properlytuned according to the Schemes presented here, it becomes an AdaptiveFuzzy Inference System (AFIS). Alternatively, the Adaptive NeuralNetwork posses an architecture to represent this FIS. In fact, thissystem constitutes a Coactive Adaptive Neuro-Fuzzy Inference System(Jang J.-S., R., Sun C.-T., Mizutani. Neuro-Fuzzy and Soft Computing,Prentice Hall Inc., Upper Saddle River, N.J. 1997). This system can alsobe properly trained off-line over a wide range of inputs according tothe Schemes provided here. This training process allows it to deal witha variety and diversity of inputs in real situations. Once the system isproperly tuned, then is ready to be used on-line at any time. Therefore,the Intelligence Inference System can be constructed and operated basedon: the Fuzzy Inference System, or as an Adaptive Fuzzy InferenceSystem, or as an Adaptive Neuro-Fuzzy Inference System. The selection ofthe construction and operation mode will depend on the intendedapplication and some implementations issues.

The function of the Expert System is to set properly the inference rulesand membership functions that are to be used by the Fuzzy InferenceSystem. The inference rules are determined by gathering Expertsknowledge available in the literature and modifying these rulesaccording to the system needs.

The following (reported) module design issues such as:

Module Structure:

Module Content:

Module Functionality:

Software System Development:

System Performance:

from some Experts can be considered to establish the inference rules.

From the collected data also the membership functions are constructed.However, they serve only as the initial set to be considered by theparameter tuning/updating process. For the adaptive neural network, theycan be considered as an initial set in the initial forward pass.Afterwards, they can be modified by subsequent backward passes requiredin the training/learning of the adaptive neural network.

The Fuzzy Logic Inference System is a Sugeno Type (several types couldbe considered, but at least a first order is necessary) with properT-norms and T-conorms (Jang J.-S. R., 1989). The considered membershipfunctions contain antecedent/premise, scaling, and consequent parametersthat can be properly tuned/updated according some appropriate Schemesprovided here. For the equivalent adaptive neural network these Schemescan be implemented such that the antecedent/premise and scalingparameters are regularly tunned by the backward pass. On the other hand,the output functions contain consequent parameters that can be updatedat the end of the forward pass. Also, here the option to incorporateweight/measures of importance for (membership functions scaling) rulediscrimination and input selection is considered.

The selection of a Sugeno Type of Fuzzy Inference System is the resultof the following fact. This type yields output functions explicitlydepending on the inputs. This feature is quite relevant under theconsidered framework. That is, this type facilitates here theidentification of a relationship between the rate of change of theoutputs with respect to the rate of change of the inputs, which isgoverned by a characteristic matrix here denoted as M.

Since the characteristic matrix M governs the system behaviour, itserves to define a novel Performance Criterion (i.e performancemeasure). Furthermore, the optimization of this criterion should ensurethat small output changes will be obtained for small input changes intothe system.

Depending on the intended application of the Intelligent Agent, therelationship between the output and the input can be considered anapproximation to an inverse function or a direct function relationship.For either case, it is easy to obtain an inverse approximation or directrelationship for the rate of change of the outputs with respect to theinputs.

If an approximation to an inverse function is considered, it is easy toshow that the characteristic matrix M is an approximation to aPseudoinverse matrix (Golub G. H., Van Loan C. F., Matrix Computations.2nd Ed. The Johns Hopkins University Press 1989). On the other hand, ifa direct relationship is considered then the characteristic matrix M isa Jacobian matrix (Golub G. H., 1989). Therefore.,the characteristicmatrix M can be considered either as a pseudoinverse, or as a Jacobianmatrix.

Moreover, if conceptually an approximation to an inverse function isconsidered, the norm of the characteristic matrix M can be selected as aPerformance Criterion. At the theoretical level this norm should satisfya consistency condition (Stewart G. W., Introduction to MatrixComputations. Academic Press. Inc., 1973), also called submultiplicativeproperty (Golub G. H. 1989). However, for practical purposes it isconvenient to consider the so-called p-norms (in particular for p=1, 2,and ∞). and the Frobenius norm (Golub G. H. 1989). The selection of thisPerformance Criterion is due to the following facts. First, notice thatin a neighborhood of the set of rank deficient matrices, for smallchanges in the input, large changes can be induced in the output. Alsonotice that the 2-norm of the pseudoinverse matrix is precisely equal tothe inverse of the smallest singular value (from the Singular ValueDecomposition (SVD) of the Jacobian). Now, since the smallest singularvalue is a measure of the distance to the set of rank deficientmatrices: the conceptual minimization of the inverse of the singularvalue will ensure a large distance to this set. Consequently, ensuringthat for small changes in the input, small changes are induced in theoutput. Notice that the 2-norm can be difficult to express explicitly.However, the Frobenius norm is a tight bound for the 2-norm, and it canbe easily expressed explicitly. Moreover, it is continuouslydifferentiable on a large domain. Therefore, it is preferable to use theFrobenius norm of the characteristic matrix M as a PerformanceCriterion.

On the other hand, if conceptually a direct function between the outputwith respect to the input is considered: then the Performance Criterioncan be set as the Frobenius norm of the characteristic matrix divided bythe sum of the Frobenius norm of the rate of change of thecharacteristic matrix with respect to each of the parameters. Thisselection is due to the fact that by setting it in this way, its minimumwill imply a neighborhood with a small bound on the condition number ofthe characteristic matrix M. Since a small condition number also impliesa situation far from the set of rank deficient matrices, the PerformanceCriterion is justified. However, in this case small changes in theoutput will imply small changes in the input.

In most applications the input space is related to a desiredperformance, whereas the output space is related to the system structureand elements that deliver that performance. Then, in most cases it ispreferably to conceptually consider that the intelligent inferencesystem is approximating an inverse function that relates the output withrespect the input. Therefore, the preferred Performance Criterion to useis the one given by the Frobenius norm of the characteristic matrix M.

The entries of the characteristic matrix M are expressed explicitly interms of the antecedent/premise, scaling, and consequent parameters.Hence, the devised Performance Criterion can be easily expressedexplicitly in terms of the antecedent/premise, scaling, and consequentparameters. Therefore an objective function can be easily defined andwhich is optimized, subject to some constraints, at the end of theforward pass of the adaptive neural network. This optimization processyields the proper updating of the consequent parameters.

In the following development a more detailed description of thedeveloped Fuzzy Inference System and some novel Learning Schemes arepresented. First, it necessary to establish a proper formulation forsome general systems relationships.

For a general system, at any instant of time, a set of variables in theplant (causal state) operational space establishes a unique set ofvariables in the task/performance (effect state) space. Formally,consider a system with n plant variables. At any instant of time, denotethese variables by ψ_(i)≡ψ_(i)(t): i=1, 2, . . . n. Also, define thetask/performance variables describing system tasks by a vector of mvariables y_(j)≡y_(j)(t): j=1,2, . . . m. Notice, that for m<n thesystem is redundant: whereas, for m>n the system is overdetermined.Also, let tε[t_(o),t_(f)] where t_(o) and t_(f) are the initial andfinal time of the task interval: and let ^(m) and ^(n) be them-dimensional and the n-dimensional Euclidean spaces respectively.Assume that y≡y(t)=[y₁,y₂, . . . , y_(m)]^(T)ε^(m) and ψ≡ψ(t)=[ψ₁,ψ₂, .. . ,ψ1_(n)]^(T)ε^(n) are related by:

y(t)=F(ψ(t))  (1)

In many applications it is desired that the task variables (effectstate) follow a reference/specified state transition, and it is requiredto calculate the corresponding plant variables (causal state). Thisimplies establishing an inverse relationship from Eq.(1). In general,this relation is nonlinear: hence an analytical inverse relationshipcannot be easily obtained. Notice that in this case, y(t) acts as aninput: whereas ψ(t) can be considered as an output. Under a localapproach the problem can be treated at the inverse functional and rateof change level. That is, the problem can be addressed in the followingindirect fashion. By differentiating Eq.(1) with respect to time, thenext equation is obtained:

{dot over (y)}(t)=J(ψ(t)){dot over (ψ)}(t)  (2)

where

{dot over (y)}≡{dot over (y)}(t)=dy(t)/dt.{dot over (ψ)}≡{dot over(ψ)}(t)=dψ(t)/dt.

and $\begin{matrix}{{{J(\psi)} \equiv {J\left( {\psi (t)} \right)}} = {\frac{\partial}{\partial\psi}{{\mathcal{F}\left( {\psi (t)} \right)}.}}} & (3)\end{matrix}$

is the (m×n) Jacobian plant matrix.

From the Eq.(2), it is possible to compute a ψ(t) state transition interms of a prescribed state transition y(t). Following an analyticalprocedure to deal with an inverse kinematics problem (Kreutz-Delgado K.,Agahi D., “A Recursive Singularity-Robust Jacobian Generalized Inverse.IEEE Trans, on Robotics and Automation. Vol. 11, No.6, December 1995),(Mayorga R. V., et al. “A Fast Approach for Manipulator InverseKinematics Evaluation and Singularities Prevention”. Journal of RoboticsSystems, Vol. 9 (8), February 1993), it can be easily shown that, form<n a general solution in an inexact context is given by:

{dot over (ψ)}=J _(wzδ) ⁺(ψ){dot over (y)}+[I−J _(wzδ) ⁺(ψ)J(ψ)]ν;  (4)

where I is the (n×n) identity matrix: ν is an arbitrary vector, intendedfor constraints compliance; and the weighted-augmented pseudoinverseJ_(wzδ) ⁺(ψ) is given by:

J _(wzδ) ⁺(ψ)=[J(ψ)^(T) ZJ(ψ)+δW] ⁻¹ J(ψ)^(T) Z:  (5)

which, it can also be expressed as

J _(wzδ) ⁺(ψ)=W ⁻¹ J(ψ)^(T) [J(ψ)W ⁻¹ J(ψ)^(T) +δZ ⁻¹]⁻¹:  (6)

where, δ>0; and the positive definite symmetric matrices W, Z, act asmetrics to allow invariance to frame reference and scaling (homogenizedimensions). Notice that the purpose of the parameter δ is to ensurewell-conditioning in Eqs.(5), and (6). Also, notice that for redundantsystems m<nj so, it is more convenient to consider the Eq.(6).Alternatively, for nonredundant systems, m≧n. ν≡0 in Eq.(4); and it ismore convenient to consider Eq.(5). Also it is important to observe thatEq.(4) turns out to be the exact solution by setting δ=0 in Eq.(5), orin Eq.(6). In general, δ=0; and it becomes activated (δ≠0) according toa robustness Scheme only in those instances in which the Jacobian matrixbecome ill-conditioned. Now, in the following developments ∥.∥,represent a standardized norm which can be expressed in terms of aweighted 2-norm. That is, in Task Space

∥{dot over (y)}∥_(s)≡({dot over (y)}^(T)Z{dot over (y)})^(½)=∥Z^(½){dotover (y)}∥₂=∥{dot over (y)}_(z)∥₂;  (7)

where, {dot over (y)}_(z)=Z^(½){dot over (y)}. Whereas, in the PlantSpace

∥{dot over (ψ)}∥_(s)≡({dot over (ψ)}^(T) W{dot over (ψ)})^(½)=∥W^(½){dot over (ψ)}∥₂=∥{dot over (ψ)}_(w)∥₂;  (8)

where,

{dot over (ψ)}_(w)=W^(½){dot over (ψ)}.

Thence, also let

∥J(ψ)∥_(s) ≡∥J _(wz)(ψ)∥_(s) ≡∥Z ^(½) J(ψ)W ^({fraction (−1/2)})∥₂;  (9)

and

∥J _(wzδ) ⁺(ψ)∥_(s) ≡∥W ^(½) J _(wzδ) ⁺(ψ)Z ^(−½)∥₂  (10)

Next observe that from Eq.(4)

{dot over (ψ)}−ν=J _(wzδ) ⁺(ψ)[{dot over (y)}−J(ψ)ν].  (11)

Thence, it is clearly observed that the proposed formulation has lead tothe development of an inverse effect-casual time-variant relationship.

Now, notice that for a general class of conventional Fuzzy InferenceSystems (FIS), such as the Sugeno Class: the system output Ω can beexplicitly expressed as a function of the antecedent/premise, scaling,and consequent parameters, and also in terms of the input variables.That is,

Ω=F ₂(α, c, u)  (12)

where, c are the consequent parameters: α are the antecedent/premise andscaling parameters: and u are the system inputs. Notice that here αincludes scaling parameters, which are usually utilized to scale themembership functions according to simple relations. These parameters canbe considered as the “importance measures” to scale the membershipfunctions using the simple relations provided in (Jang J.-S., 1997).Notice that this expression can also be expressed as:

Ω=M _(o)(α, c, u)u:  (13)

where, M_(o) is an (n×m) matrix. This class uses Eq.(12) with fixedparameters and has been successful for some Control Problems (JangJ.-S., 1997). However, the Adaptive Fuzzy Inference Systems (AFIS) andCo-Active Neuro-Fuzzy Inference Systems (CANFIS), that include Schemesfor tuning/updating the parameters, have demonstrated a superior/betterperformance (Jang J.-S., 1997).

In fact, one of the most important and distinguishing characteristics ofAFIS and CANFIS, is the ability to tune/update the parameters forLearning purposes. Usually, the construction of conventional AFIS andCANFIS is based on approaches that establish some standard errormeasures from which some proper tuning/updating Schemes are developed.Moreover, these procedures have been conceptually devised for modellingor approximating direct functions. Unlike these conventional approaches,here a novel Performance Criterion is presented and utilized to developnovel Schemes for parameter tuning/updating. Furthermore, the approachconsidered here has been conceptually devised to approximate inversefunctions.

Now, observe that from Eq.(12), it follows that

{dot over (Ω)}=J ₂(α, c, u){dot over (u)};  (14)

where, J₂=∂F₂/∂u. For notation purposes let the (n×m) matrix M(α, c,u)≡J₂(α, c, u): thence,

{dot over (Ω)}=M(α, c, u){dot over (u)}.  (15)

As will be appreciated by those skilled in the art, the matrix M (α, c,u) contains one or more elements of α, c, u. Now, let the output of theinference system be related to the plant variables as follows:

{dot over (Ω)}≡{dot over (ψ)}ν;  (16)

whereas, the input of the inference system be related to the taskvariables:

{dot over (u)}≡{dot over (y)}−J(ψ)ν:  (17)

thence, by Eq.(15) it follows that

{dot over (ψ)}−ν=M=M(α, c, u)[{dot over (y)}−J(ψ)ν].  (18)

Then, clearly from Eqs.(11) and (18) it can be easily observed that

M(α, c, u)J _(wzδ) ⁺(ψ):  (19)

that is, the matrices M(α, c, u) and J_(wzδ) ⁺(ψ) are directly related.Then, in practice for the considered FIS class, the matrix M(α, c, u)should be constructed in some way such that

M(α, c, u)→J _(wzδ) ⁺(ψ).  (20)

In general, for Control problems there is available a desired orreference (optimal) solution, and an error measure is usually defined interms of the difference between the desired or reference output and theactual system output. This error measure is normally used by AFIS andCANFIS for Learning purposes by properly optimizing the error measureaccording to some parameter tuning/updating Schemes (resulting in anappropriate shifting and shaping of the membership functions).

However, the application of FIS for Decision Making or DecisionResolution problems is not as simple. This is due to the fact that it isnot easy to justify a priori a desired nor optimal output vector. Atmost the output may incorporate some experts knowledge. Then, since itis difficult to justify a priori a desired or optimal output vector, itis also difficult to establish a priori an error measure to deviseschemes for proper parameter tuning/updating. Therefore, theconstruction and application of AFIS or CANFIS for Decision Making orDecision Resolution problems becomes more complicated, mainly due to theadditional difficulty in establishing an error measure.

For the general class of FIS considered here, a possible way to dealwith the difficulty is to establish as a target for optimization aPerformance Criterion, rather than an error measure. Here, Eq.(19)precisely serves this purpose and leads to the development of theproposed approach. First, all important property of J_(wzδ) ⁺(ψ) isexploited. It is well known in the literature that the norm (asconsidered here) of this matrix determines the system behaviour. FromEq.(11) it easily follows that:

∥{dot over (ψ)}−′ν∥≦∥J _(wzδ) ⁺(ψ)∥ ∥{dot over (y)}−J(ψ)ν∥.  (21)

That is, the smaller the value that ∥J_(wzδ) ⁺(ψ)∥ takes, the smaller isthe upper bound of the norm of the output {dot over (ψ)}−ν it.Therefore, in view of the Eq.(19), the norm of the matrix M constitutesa meaningful Preformance Criterion.

Thence, here two main issues are considered:

the Optimization of the Performance Criterion: that is, the Minimizationof the norm of M: and

to ensure that M→J_(wzδ) ⁺, by establishing a proper constraint for theOptimization process.

In order to develop a proper strategy for parameter tuning updating itis necessary to consider the following development. For simplicitypurposes, let's drop the subindex s from ∥.∥, in the subsequentexpressions.

First, let's define (refer to FIG. 3) the actual task transitioncorresponding to the output {dot over (Ω)} as:

{dot over (u)} _(α) ≡J _(E){dot over (Ω)};  (22)

where J_(E) is an approximation to the Jacobian matrix J. Thence, byEq.(15)

{dot over (u)} _(α) =J _(E) M(α, c, u){dot over (u)}.  (23)

Then, the difference between it desired or specified task transition,and the actual one obtained from the output of the inference system isgiven by $\begin{matrix}{{ - {\overset{.}{}}_{\alpha}} = {\overset{.}{} - {J_{E}{M\left( {\alpha \cdot c \cdot } \right)}\overset{.}{}}}} & {\quad (24)} \\{= \quad {\left\lbrack {I - {J_{E}{M\left( {\alpha \cdot c \cdot } \right)}}} \right\rbrack {\overset{.}{}.}}} & {\quad (25)}\end{matrix}$

Therefore,

∥{dot over (u)}−{dot over (u)} _(α) ∥/∥{dot over (u)}∥≦∥I−J _(E) M(α, c,u)∥.  (26)

Now, notice that $\begin{matrix}{{I - {J_{E}{M\left( {\alpha \cdot c \cdot } \right)}}} = {J_{E}\left\lbrack {J_{E}^{+} - {M\left( {\alpha \cdot c \cdot } \right)}} \right\rbrack}} & {\quad (27)} \\{= {{\left\lbrack {{M\left( {\alpha \cdot c \cdot } \right)}^{+} - J_{E}} \right\rbrack {M\left( {\alpha \cdot c \cdot } \right)}}:}} & {\quad (28)}\end{matrix}$

where the (n×m) matrix J_(E) ⁺, is the pseudoinverse of J_(E); and the(m×n) matrix M(α, c, u) is the pseudoinverise of M(α, c, u). Then, iteasily follows that

∥I−J _(E) M(α, c, u)∥≦∥J _(E) ∥ ∥J _(E) ⁺ −M(α, c, u).∥;  (29)

which can also be expressed as

∥i−J _(E) M(α, c, u)∥≦∥M(α, c, u)⁺ −J _(E) ∥ ∥M(α, c, u)∥.  (30)

In general, for any system it is required that

∥{dot over (u)}−{dot over (u)} _(α) ∥/∥{dot over (u)}∥→0.  (31)

By Eqs.(26), (29), and (30), this condition can be easily satisfied byany of the following expressions:

∥I−J _(E) M(α, c, u)∥→0;  (32)

∥J _(E) ∥ ∥J _(E) ⁺ −M(α, c, u)∥→0:  (33)

∥M(α, c, u)⁺ −J _(E) ∥ ∥M(α, c, u)∥→0.  (34)

Obviously, if the Jacobian matrix J is known and available; thenJ_(E)≡J, and J_(E) ⁺=J⁺ in these expressions. However, in order toprevent ill-behaviour for those instances in which the Jacobian becomesill-conditioned, it is preferable to set J_(E) ⁺=J_(wzδ) ⁺. This is dueto the fact that when the Jacobian becomes ill-conditioned ∥J⁺∥ becomesexcessively large (inducing an excessively large output): whereas,∥J_(wzδ) ⁺∥ remains bounded.

Notice that Eqs.(32), (33), and (34), also serve to establish acondition or constraint to ensure that M(α, c, u)→J_(E) ⁺→J_(wzδ) ⁺(ψ).It is important to notice that expression (32) or (33) can be useful forboth Control Problems, and Decision (Making/Resolution) Problems.However, for Decision problems, it is more difficult to estimate theJacobian matrix. Thence, for this case it is more convenient to considerthe expressions (34) and (32) in an appropriate way. This fact leads todevise somewhat different Schemes for Control Problems than those forDecision Problems. However, in both cases the developed Schemes relyheavily on the optimization of the Performance Criterion.

The nature of the matrix M(α, c, u) and Eqs.(32), (33), (34), lead toconsider an off-line Optimization process as a strategy to tune/updateparameters. That is, the ∥M(α, c, u)∥ can be considered as an objectivefunction, and the Eqs.(32), (33), (34), are utilized to establish someappropriate constraints. Then, the parameters are tuned/updated by anOptimization process consisting of solving a series of Minimizationproblems. However, in order to ensure obtaining a global optimum foreach Minimization problem is more convenient to consider the convexfunction ∥M(α, c, u)∥² as an objective function to be Minimized.

Also, notice that the Minimization of ∥M(α, c, u)∥² can be performedwith respect to α, c, and/or u: subject to some constraints fromEqs.(32), (33), (34): and also subject to a set of some appropriate(usually provided by the user) constraints. This leads to a solutionwhich is optimal with respect to that set of constraints. However, inorder to allow generality for a diversity of inputs, and facilitate theOptimization process is preferable to perform this process in two stagesconsisting of an initial Procedure and a parameter tuning/updatingScheme. As shown next, an initial Procedure and several Schemes can bedevised both for Control Problems as well as for Decision Problems.

The FIGS. 4a-4 e, show an overview of the envisioned Procedures andSchemes. Now, let T ol>0 be a small scalar representing a tolerancefactor. Also, let η>0 be a scalar which can be used to normalize thesystem inputs: that is, η=1, or if appropriate set to another value. ForControl Problems the following initial Procedure is first performed:

Procedure A

Select an appropriate set of initial values α_(o), c_(o), and u_(o) forthe Minimization process.

Minimize ∥M(α, c, u)∥² with respect to α, c, and u: and subject to someappropriate constraints on α, c, and u. Obtain the optimized α*, c*, andu*.

From Eq.(13) obtain the corresponding, Ω(α*, c*, u*): and computeψ*=Ω(α, c, u,)+νΔt, where ν is a specified vector for constraintcompliance, and Δt>0 is a small scalar.

Compute J_(E)(ψ*).

Set the initial values α=α*, and c_(o)=c*, for the next Minimizationprocess: or, if appropriate, select another set of initial values.

Set {tilde over (α)}=α*, and {tilde over (c)}=c*: or, if appropriate,select another set of values.

Set u=ũ such that ∥ũ∥=η.

Once this initial Procedure is performed, any of the next four Schemescan be selected for proper parameter tuning/updating.

Scheme I

Minimize ∥M(α, c, u)∥² with respect to the antecedent/premise andscaling parameters α and the consequent parameters c: and subject to∥I−J_(E)(ψ*)M(α, c, ũ)∥²≦T ol, and some appropriate constraints on α,and c. Obtain and record {tilde over (α)}, and c, and also record∥M({tilde over (α)}, {tilde over (c)}, ũ)∥.

Keep the initial values αhd o=α*, and c_(o)=c*: or, if appropriate, setα_(o)={tilde over (α)}, c_(o)={tilde over (c)}: or select anothersuitable set of initial values.

Repeat procedure for a specified number of iterations, for a wide rangeof u=ũ, such that ∥ũ∥=η.

Select among the recorded values the smallest ∥M({tilde over (α)},{tilde over (c)}, ũ)∥, and chose as optimal parameters the corresponding{tilde over (α)}, and {tilde over (c)}.

Notice that this Scheme requires only one Minimization per iteration.However, in order to facilitate the use of Neuro-Fuzzy InferenceSystems: it is convenient to split the Optimization process in twostages per iteration as shown in the following three Schemes.

Scheme II

Set the antecedent/premise and scaling parameters α={tilde over (α)}.

Minimize ∥M({tilde over (α)}, c, ũ)∥² with respect to the consequentparameters c: and subject to ∥I−J_(E)(ψ*)M({tilde over (α)}, c, ũ)∥²≦Tol, and some appropriate constraints on c. Obtain {tilde over (c)}.

Set c={tilde over (c)}, and Minimize ∥M(α, {tilde over (c)}, ũ)∥² withrespect to the antecedent/premise and scaling parameters α: and subjectto ∥I−J_(E)(ψ*)M(α, {tilde over (c)}, ũ)∥²≦T ol, and some appropriateconstraints on α. Obtain and record {tilde over (α)}: and also record{tilde over (c)}, and ∥M({tilde over (α)}, {tilde over (c)}, ũ)∥,

keep the initial values α_(o)=α*, and c_(o)=c*: or, if appropriate, setα_(o)={tilde over (α)}, c_(o)={tilde over (c)}; or select anothersuitable set of initial values. Also, if appropriate, keep the obtainedvalues {tilde over (α)}: otherwise set them to suitable values.

Repeat procedure for at specified number of iterations, for a wide rangeof u=ũ, such that ∥ũ∥=η.

Select among the recorded values the smallest ∥M({tilde over (α)},{tilde over (c)}, ũ)∥, and chose as optimal parameters the corresponding{tilde over (α)}, and {tilde over (c)}.

A variant of this Scheme, can easily be developed and is given asfollows:

Scheme III

Set the consequent parameters c={tilde over (c)}.

Minimize ∥M(α,{tilde over (c)}, ũ)∥² with respect to theantecedent/premise and scaling parameters α: and subject to∥I−J_(E)(ψ*)M(α, {tilde over (c)}, ũ)∥²≦T ol, and some appropriateconstraints on α. Obtain {tilde over (α)}.

Set α={tilde over (α)}, and Minimize ∥M({tilde over (α)}, c, ũ)∥² withrespect to the consequent parameters c: and subject to∥I−J_(E)(ψ*)M({tilde over (α)}, c, ũ)∥²≦T ol, and some appropriateconstraints on c. Obtain and record {tilde over (c)}: and also record{tilde over (α)}, and ∥M({tilde over (α)}, {tilde over (c)}, ũ)∥.

Keep the initial values α_(o)=α*, and c_(o)=c*: or, if appropriate, seta_(o)={tilde over (α)}, c_(o)={tilde over (c)}; or select anothersuitable set of initial values. Also, if appropriate, keep the obtainedvalues {tilde over (c)}; otherwise set them to suitable values.

Repeat procedure for a specified number of iterations, for a wide rangeof u=ũ, such that ∥ũ∥=η.

Select among the recorded values the smallest ∥M({tilde over (α)},{tilde over (c)}, ũ)∥, and chose as optimal parameters the corresponding{tilde over (α)}, and {tilde over (c)}.

The previous two Schemes facilitate the use of Neuro-Fuzzy InferenceSystems. However, in some cases it is required to minimize the error ofthe system output with respect to an optimal or reference output: i.e.with respect to Ω(α*, c*, u*). In these cases it is convenient tobenefit from the fact that the output Ω(α, c, u) can be expressed as alinear function of the consequent parameters c, and then use a LeastSquares Minimization as a first stage. Thence, the next Scheme can beeasily devised.

Scheme IV

Set the antecedent/premise and scaling parameters α={tilde over (α)}.

For a wide range of of ũ such that ∥ũ∥=η: set Ω({tilde over (α)}, c,ũ)=Ω(α*, c*, u*) and solve by a Least Squares method for the consequentparameters c. Obtain {tilde over (c)}.

Set c={tilde over (c)}, and for any ũ of the considered range; Minimize∥Ω(α, {tilde over (c)}, ũ)−Ω(α*, c*, u*)∥² with respect to theantecedent/premise and scaling parameters α: subject to some appropriateconstrains on α, and ∥I−J_(E)(ψ*)M(α, {tilde over (c)}, ũ)∥²≦T ol.Obtain and record {tilde over (α)}: and also record {tilde over (c)},and D≡∥Ω({tilde over (α)}, {tilde over (c)}, ũ)−Ω(α*, c*, u*)∥².

Keep the initial values α_(o)=α*, and c_(o)=c*: or, if appropriate setα_(o)={tilde over (α)}, c_(o)={tilde over (c)}; or select anothersuitable set of initial values. Also, if appropriate, keep the obtainedvalues {tilde over (α)}: otherwise set them to suitable values.

Repeat procedure, for a specified number of iterations.

Select among the recorded values the smallest D, and chose as optimalparameters the corresponding {tilde over (α)}, and {tilde over (c)}.

Since for Decision Problems it is not easy to estimate the Jacobianmatrix J_(E)(ψ), the parameter tuning/updating takes somewhat differentstages. According to Eq.(34), the Minimization of ∥M(α,c, u)∥ is firstsought: then, J_(E) is set to J_(E)≡M(α*, c*, u*)⁺; and finally theEq.(32) is used. This leads to the following initial Procedure and fourSchemes:

Also in this case, let T ol>0 be a small scalar representing a tolerancefactor, and the following initial Procedure is first performed:

Procedure B

Select an appropriate set of initial values α_(o), c_(o), and u_(o) forthe Minimization process.

Minimize ∥M(α, c, u)∥² with respect to α, c, and u: and subject to someappropriate constraints on α, c, and u. Obtain the optimal α*, c*, andu*.

Obtain M(α*, c*, u*) and compute numerically M(α*, c*, u*)⁺.

Set the initial values α_(o)=α*, and c_(o)=c*, for the next Minimizationprocess, or, if appropriate, select another suitable set of initialvalues.

Set {tilde over (α)}=α*, and {tilde over (c)}=c*: or, if appropriate,select another appropriate set of values.

Set u=ũ such that ∥ũ∥=η.

Once this initial Procedure is performed, any of the next four Schemescan be selected for proper parameter tuning/updating.

Scheme V

Minimize ∥M(α, c, ũ)∥² with respect to the antecedent/premise andscaling parameters α and the consequent parameters c: and subject to∥−M(α*, c*, u*)⁺M(α, c, ũ)∥²≦T ol, and some appropriate constrains on α,and c. Obtain and record {tilde over (α)}, and {tilde over (c)}, andalso record ∥M({tilde over (α)}, {tilde over (c)}, ũ)∥,

keep the initial values α_(o)=α*, and c_(o)=c*: or, if appropriate, setα_(o)={tilde over (α)}, c_(o)={tilde over (c)}; or select anothersuitable set of initial values.

Repeat procedure for a specified number of iterations, for a wide rangeof u=ũ, such that ∥ũ∥=η.

Select among the recorded values the smallest ∥M({tilde over (α)},{tildeover (c)}, ũ)∥, and chose as optimal parameters the corresponding {tildeover (α)}, and {tilde over (c)}.

Again, notice that this Scheme requires only one Minimization periteration. Also for Decision Problems, in order to facilitate the use ofNeuro-Fuzzy Inference Systems: it is convenient to split theOptimization process in two stages per iteration as shown in thefollowing three Schemes.

Scheme VI

Set the antecedent/premise and scaling parameters α={tilde over (α)}.

Minimize ∥M({tilde over (α)}, c, ũ)∥² with respect to the consequentparameters c: and subject to ∥I−M(α*, c*, u*)⁺M({tilde over (α)}, c,ũ)∥²≦T ol, and some appropriate constraints on c. Obtain {tilde over(c)}.

Set c={tilde over (c)} and Minimize ∥M(α, {tilde over (c)}, ũ)∥² withrespect to the antecedent/premise and scaling parameters α: and subjectto ∥I−M(α*, c*, u*)⁺M(α, {tilde over (c)}, ũ)∥² ≦T ol, and someappropriate constraints on α. Obtain and record {tilde over (α)}; andalso record {tilde over (c)}, and ∥M({tilde over (α)}, {tilde over (c)},ũ)∥.

Keep the initial values α_(o)=α* , and c_(o)=c*: or, if appropriate, setα_(o)={tilde over (α)}, c_(o)={tilde over (c)}; or select anothersuitable set of initial values. Also, if appropriate, keep the obtainedvalues {tilde over (α)}: otherwise set them to suitable values.

Repeat procedure for a specified number of iterations, for a wide rangeof u=ũ, such that ∥ũ∥=η.

Select among the recorded values the smallest ∥M({tilde over (α)},{tilde over (c)}, ũ)∥, and chose as optimal parameters the corresponding{tilde over (α)}, and {tilde over (c)}.

A variant of this Scheme, can easily be developed and is given asfollows:

Scheme VII

Set the consequent parameters c={tilde over (c)}.

Minimize ∥M(α, {tilde over (c)}, ũ)∥² with respect to theantecedent/premise and scaling parameters α: and subject to ∥I−M(α*, c*,u*)⁺M(α,{tilde over (c)}, ũ)∥² ≦T ol, and some appropriate constraintson α. Obtain {tilde over (α)}.

Set α={tilde over (α)}, and Minimize ∥M({tilde over (α)}, c, ũ)∥² withrespect to the consequent parameters c, and subject to ∥I−M(α*, c*,u*)⁺M({tilde over (α)}, c, ũ)∥²≦T ol, and some appropriate constraintson c. Obtain and record {tilde over (c)}; and also record {tilde over(α)}, and ∥M({tilde over (α)}, {tilde over (c)}, ũ)∥.

Keep the initial values α_(o)=α*, and c_(o)=c*: or, if appropriate, setα_(o)={tilde over (α)}, c_(o)={tilde over (c)}: or select anotherappropriate set of initial values. Also, if appropriate, keep theobtained values {tilde over (c)}: otherwise set them to suitable values.

Repeat procedure for a specified number of iterations, for a wide rangeof u=ũ, such that ∥ũ∥=η.

Select among the recorded values the smallest ∥M({tilde over (α)},{tilde over (c)}, ũ)∥, and chose as optimal parameters the corresponding{tilde over (α)}, and {tilde over (c)}.

Again, the previous two Schemes facilitate the use of Neuro-FuzzyInference Systems. However, in some cases (in which it is required tominimize the error of the system output with respect to an optimal orreference output) it is convenient to benefit from the fact that theoutput Ω(α, c, u) can be expressed as a linear function of theconsequent parameters, and then use a Least Squares Minimization as afirst stage. Notice that for Decision Problems although there is no aprior optimal nor reference output, the Procedure B allows to considerΩ(α*, c*, u*) as an appropriate reference output. Thence, the nextScheme can be easily devised.

Scheme VIII

Set the antecedent/premise and scaling parameters α={tilde over (α)}.

For a wide range of wide range of ũ such that ∥ũ∥=η: set Ω({tilde over(α)}, c, ũ)=Ω(α*, c*, u*) and solve by a Least Squares method for theconsequent parameters c. Obtain {tilde over (c)}.

Set c={tilde over (c)}, and for any ũ of the considered range; Minimize∥Ω(α, {tilde over (c)}, ũ)−Ω(α*, c*, u*)∥² with respect to theantecedent/premise and scaling parameters α: subject to and someappropriate constraints on α, and ∥I−M(α*, c*, u*)⁺M(α{tilde over (c)},ũ)∥² ≦T ol. Obtain and record {tilde over (α)}: and also record {tildeover (c)}, and D≡∥Ω({tilde over (α)}, {tilde over (c)}, ũ)−Ω(α*, c*,u*)∥².

Keep the initial values α_(o)=α*, and c_(o)=c*: or, if appropriate, setα_(o)={tilde over (α)}, c_(o)={tilde over (c)}; or select anothersuitable set of initial values. Also, if appropriate, keep the obtainedvalues {tilde over (α)}: otherwise set them to suitable values.

Repeat procedure, for a specific number of iterations.

Select among the recorded values the smallest D, and chose as optimalparameters the corresponding {tilde over (α)}, and {tilde over (c)}.

Recall that that the expression ∥.∥ stands for the s-norm ∥.∥_(s), whichby Eqs.(7), (8). and (9) can be expressed ill terms of the weighted2-norm ∥.∥₂. Although appropriate, the 2-norm is a bit difficult to workwith. Since the Frobenius norm represents a tight upper bound for the2-norm, and is continuously differentiable in a large domain; thence, itis more convenient to consider the Frobenius norm in the Optimizationprocess. That is, consider the properly weighted (n×m matrix) B: hence,its Frobenius norm is given by

∥B∥ _(F)≡[Σ_(i=1) ^(n)Σ_(j=1) ^(m) |b _(ij)|²]^(½):  (35)

where, for i=1, . . . n:j=1, . . . m: b_(ij) is the (i, j) component ofthe weighted matrix B. Thence, for implementation purposes is moreconvenient to consider in the Optimization Process ∥W^(½)M(α, c,u)Z^(−½)∥_(F); and ∥Z^(½)[I−AM(α, c, u)Z^(−½)∥_(F), where A=J_(E)(ψ*),or A=M(α*, c*, u*): and the Frohenius norm of other pertinentexpressions.

Notice that in some applications the knowledge from the Experts(embedded in the membership functions), and following the Procedure A orB, may be sufficient for a somewhat effective system preformance.Thence, the considered Class of Fuzzy Inference Systems (FIS) can beconsidered as the Intelligent Inference System. However, in someapplications it is required a high level of performance. In those casesis necessary to utilize also the previous Schemes.

A convenient strategy for parameter tuning/updating can consist of a twostage process: (a) perform an initial Procedure: and (b) then followwith any of the previous Schemes. However, in some cases it may beconvenient to devise a strategy consisting of several stages. That is,(a) perform an initial procedure: (b) follow a particular Scheme up to acertain point: and then (continue (c) with a another Scheme. Moreover,for Decision Problems, in some cases it may be convenient to start withthe Procedure B: then, follow one or several of Schemes V-VIII: and,once an approximation to the Jacobian matrix J_(E) is available,continue with anyone of the Schemes II to IV.

Again, notice that most of the previous Schemes have been devised basedon splitting the Optimization process in some Minimization stages. Thisis done to facilitate the implementation of the Adaptive Neuro-FuzzyInference System proposed here.

It is important to mention that the implementation of anyone of theseSchemes enables the considered Class of Fuzzy Inference Systems (FIS) tobecome in fact a Class of Adaptive Fuzzy Inference Systems (AFIS). Inmany applications this class of Adaptive Fuzzy Inference Systemssuffices to construct the Intelligent Inference System. However, in someapplications the Schemes may require to perform a lengthly Optimizationprocess. Thence, in those cases then it is preferable to utilize theenvisioned Adaptive Neuro-Fuzzy Inference System described next.

Notice that a class of Adaptive Networks are functionally equivalent tosome Fuzzy Inference Systems. It is relatively easy to show (Jang J.-S.,1997) that with an appropriate architecture this class of AdaptiveNetworks can represent the Class of Fuzzy Inference Systems consideredhere.

Usually for Adaptive Networks the training process is based inperforming a series of forward and backward passes. The Class ofAdaptive Networks envisioned here posses an architecture containing theappropriate number of layers to represent the Class of Fuzzy InferenceSystems being considered here. In order to facilitate the hybridlearning of this Class of Adaptive Networks the parameter set isproperly decomposed and several of the previous Schemes (for ControlProblems II, III, or IV: whereas for Decision Problems VI, VII, VIII)devised in the previous section can be easily implemented. The resultantinference system in fact constitutes an Adaptive Neuro-Fuzzy InferenceSystem.

That is, here also the overall training process consist of a series offorward and backward passes according to the( selected Scheme. For,example, according to Scheme II (or VI), in the forward pass theantecedent premise and scaling parameters are held constant, and at theend of the pass the consequent parameters are updated according to thecorresponding Scheme step. Then, the consequent parameters are heldconstant and the antecedent/premise and scaling parameters are updatedby the gradient descent according to the corresponding Scheme step. Ifnecessary, for fine tunning/updating purposes, this process can befollowed by another Scheme.

As previously mentioned, unlike other current applications, theobjective is not to reduce it target error with respect at conventionalreference output: but rather to optimize the proposed PerformanceCriterion over a wide range of inputs. That is, here theantecedent/premise and scaling parameters are properly tuned/updated,and the membership functions modified accordingly. Here, also theconsequent parameters are properly tuned/updated to ensure that smalloutput changes correspond to small input changes.

Notice that here the tuning/updating of the membership functions isbased on tuned and updated antecedent/premise, scaling, and consequentparameters resulting from the optimization of a meaningful PerformanceCriterion. Therefore, the usual inherent subjectivity carried from theExperts into the membership functions is reduced. This fact isparticular significant for Decision Problems.

The output of the Intelligent Inference System is a decisionmaking/resolution set and diagnostic on the proper relations/percentagesof structural elements and components and indicators of geometricalarrangements that constitute a proper design. Among others thesecomponents can be:

Text:

Video;

Film;

Audio;

Static/Dynamic Graphics:

Images/Pictures:

Browser features:

Linkages maps

Figures, frames, icons, etc.

The output can be expressed and implemented in a linguistic form, toserve as a guide for the design of the module to be displayed: or usedto create, refine, and display automatically the module.

The Output Interface consists of a software program that receives somecommand tags from the Coordinating Module, and the decision resolution(diagnostic) from the Intelligent Inference System. In a generation modeit translates them into tags/labels (values) that are incorporated andproperly set in a generated (advanced Hypertext Language type such as,but not necessarily, the HTML: or the Extensible Markup Language XML:and that allow dynamic interaction and operation) descriptive module. Inthe retrieval mode, similar steps are taken. However, in this case thereis no actual descriptive module generation: rather, the target module isthe retrieved descriptive module to which the updated tags/labels(values) are just transferred.

Finally, once the tags/label (values) are properly set (in thegenerated, or retrieved Hypertext Language type unit): the module can beinstantaneously displayed.

FIG. 2 illustrates a computer which may use the described intelligentagent. Turning to FIG. 2, the computer comprises a processor 30, memory31, a user interface including a display 32 and speaker 34, and a mediareader 36. Media reader 36 may read a computer media product 38 storingthe described intelligent agent. In operation, the intelligent agent onmedia product 38 may be loaded into memory 31 and executed by processor30 is aforedescribed in order to control the display and speaker whenexecuting display and audio software modules.

It will be apparent that the described FIS and ANFIS models with tunedparameters could be used for any control problem or decision makingproblem. It is particularly advantageously employed in connection withhighly non-linear target systems.

Modifications will be apparent to those skilled in the art and,therefore, the invention is defined in the claims.

What is claimed is:
 1. A method of operating a target system, comprisingthe steps of: (a) at each of a series of instants in time, developingone of a fuzzy inference system (FIS) model and an adaptive neuro-fuzzyinference system model of said target system, comprising: modelling arate of change of an output vector of said target system as a product ofa matrix of elements and a rate of change of an input vector of saidsystem, elements of said matrix dependent upon antecedent, scaling, andconsequent parameters and at least one element from said input vector;receiving constraints on antecedent, scaling, and consequent parametersand input vectors, thereby defining sets of acceptable antecedent,scaling, and consequent parameters and input vectors; receiving initialantecedent, scaling, and consequent parameters and an initial inputvector; minimizing a consistent norm of said matrix to obtain optimizedantecedent, scaling, and consequent parameters; and (b) operating saidtarget system based on said developed model.
 2. The method of claim 1wherein said step of minimizing a norm of said matrix comprisesminimizing a square of a norm of said matrix.
 3. The method of claim 2wherein said step of minimizing a square of a norm of said matrixcomprises minimizing a square of a Frobenious norm of said matrix. 4.The method of claim 2 wherein said step of minimizing a norm of saidmatrix also obtains an optimized input vector and including the step ofobtaining an estimated Jacobian matrix based on said optimizedantecedent, scaling, and consequent parameters and said optimized inputvector.
 5. The method of claim 4 including the steps of: constrainingacceptable input vectors to those having a norm equivalent to aconstant; repetitively: (a) selecting a fixed input vector meeting saidconstraining step; (b) selecting initial antecedent, scaling, andconsequent parameters; (c) minimizing a square of a norm of said matrixto obtain a valued matrix and corresponding antecedent, scaling, andconsequent parameters; (d) provided a square of a norm of a differenceof (i) an identity matrix and (ii) a product of (A) said estimatedJacobian matrix and (B) said valued matrix is less than a pre-settolerance, storing a norm of said valued matrix and said correspondingantecedent, scaling, and consequent parameters; selecting a stored normof said valued matrix having a smallest value and selecting antecedent,scaling, and consequent parameters corresponding with said selectedstored norm as tuned antecedent, scaling, and consequent parameters. 6.The method of claim 5 wherein step (b) comprises, on a first iteration,selecting said optimized antecedent, scaling, and consequent parametersas initial antecedent, scaling, and consequent parameters.
 7. The methodof claim 6 wherein step (b) comprises, on an iteration subsequent tosaid first iteration, selecting antecedent, scaling, and consequentparameters stored from step (d) of a prior iteration as initialantecedent, scaling, and consequent parameters.
 8. The method of claim 7wherein said step of constraining acceptable input vectors comprisesconstraining acceptable input vectors to those having a norm equal to apreset value.
 9. The method of claim 5 wherein steps (c) and (d) areundertaken twice each iteration, once with antecedent and scalingparameters fixed at their initial values to obtain consequent parameterswhich are stored in a first repetition of step (d) and a second timewith consequent parameters fixed at their obtained values to obtainantecedent and scaling parameters and a norm of a corresponding valuedmatrix which are stored in a second repetition of step (d).
 10. Themethod of claim 5 wherein steps (c) and (d) are undertaken twice eachiteration, once with consequent parameters fixed at their initial valuesto obtain antecedent and scaling parameters which are stored in a firstrepetition of step (d) and a second time with antecedent and scalingparameters fixed at their obtained values to obtain consequentparameters and a norm of a corresponding valued matrix which are storedin a second repetition of step (d).
 11. The method of claim 5 whereinsaid estimated Jacobian comprises a generalized inverse of a matrixobtained by said minimization step of claim
 1. 12. The method of claim 5wherein said target system comprises a user interface and wherein saidstep of receiving constraints on antecedent, scaling, and consequentparameters and input vectors and the step of receiving initialantecedent, scaling, and consequent parameters and an initial inputvector comprises: receiving a rule-based expert system for said userinterface with initial membership functions; and receiving userpreferences for said user interface.
 13. The method of claim 4 includingthe steps of: determining an optimized output vector based on saidoptimized antecedent, scaling, and consequent, parameters; repetitively:selecting initial antecedent and scaling parameters; selecting a widerange of input vectors meeting said constraining step; for each one ofsaid selected wide range of input vectors, set an expression for anoutput vector which is a function of said one input vector, said initialantecedent and scaling parameters, and unknown consequent parametersequivalent to said optimized output vector and determine consequentparameters based on a least squares method; minimize a square of a normof a difference of (A) an output vector which is a function of unknownantecedent and scaling parameters, said determined consequentparameters, and any one input vector of said selected wide range ofinput vectors and (B) said optimized output vector to obtain a minimizedvalue and corresponding antecedent and scaling parameters; provided asquare of a norm of a difference of (i) an identity matrix and (ii) aproduct of (A) said estimated Jacobian matrix and (B) said valued matrixis less than a pre-set tolerance, storing a minimized value from saidminimizing step and said corresponding antecedent, scaling, andconsequent parameters; selecting a stored minimized value having asmallest value and selecting antecedent and consequent parameterscorresponding with said selected stored value as tuned antecedent andconsequent parameters.
 14. The method of claim 1 wherein said targetsystem comprises a user interface and wherein said step of receivingconstraints on antecedent, scaling, and consequent parameters and inputvectors and the step of receiving initial antecedent and consequentparameters and an initial input vector comprises: receiving a rule-basedexpert system for said user interface with initial membership functions;and receiving user preferences for said user interface.
 15. A method ofdeveloping a fuzzy inference system (FIS) model of a target system foruse in operating said target system, comprising the steps of: modellinga rate of change of an output vector of said target system as a productof a matrix of elements and a rate of change of an input vector of saidsystem, elements of said matrix dependent upon antecedent, scaling andconsequent parameters and at least one element from an input vector;receiving constraints on antecedent, scaling, and consequent parametersand input vectors, thereby defining sets of acceptable antecedent,scaling and consequent parameters and input vectors; receiving initialantecedent, scaling, and consequent parameters and an initial inputvector; and minimizing a norm of said matrix to obtain optimizedantecedent, scaling, and consequent parameters.
 16. An intelligent agentfor use in operating a user interface comprising: an active inputinterface for presenting user preference questions and receiving userpreferences; an expert system for said user interface with initialmembership functions; a fuzzy logic inference system (FIS); an adaptiveneural network; and an output interface for controlling said userinterface; said FIS and adaptive neural network for receivingconstraints on antecedent, scaling, and consequent parameters and inputvectors, thereby defining sets of acceptable antecedent, scaling, andconsequently parameters and input vectors and for receiving initialantecedent, scaling and consequent parameters and an initial inputvector from said active input interface and said expert system and forfor each of a series of instants in time: modeling a rate of change ofan output vector of said user interface as a product of a matrix ofelements and a rate of change of an input vector of said system,elements of said matrix dependent upon antecedent, scaling, andconsequent parameters and at least one element from an said inputvector; and minimizing a norm of said matrix to obtain optimizedantecedent, scaling, and consequent parameters and an optimized inputvector.
 17. The intelligent agent of claim 16 wherein said FIS andadaptive neural network are also for: obtaining an estimated Jacobianmatrix based on said optimized antecedent, scaling, and consequentparameters and said optimized input vector; constraining acceptableinput vectors to those having a norm equivalent to a constant;repetitively: (a) selecting a fixed input vector meeting saidconstraining step; (b) selecting initial antecedent, scaling, andconsequent parameters; (c) minimizing a square of a norm of said matrixto obtain a valued matrix and corresponding antecedent, scaling, andconsequent parameters; (d) provided a square of a norm of a differenceof (i) an identity matrix and (ii) a product of (A) said estimatedJacobian matrix and (B) said valued matrix is less than a pre-settolerance, storing a norm of said valued matrix and said correspondingantecedent, scaling, and consequent parameters; and selecting a storednorm of said valued matrix having a smallest value and selectingantecedent and consequent parameters corresponding with said selectedstored norm as tuned antecedent and consequent parameters.
 18. Theintelligent agent of claim 17 wherein said FIS and adaptive neuralnetwork, in obtaining said estimated Jacobian, obtain a generalizedinverse of a matrix of elements obtained in said modelling operation.19. A computer readable medium storing computer executable instructionsthat when loaded by a computing device, adapt said computing device to:provide an active input interface for presenting user preferencequestions and receiving user preferences; provide an expert system forsaid user interface with initial membership functions; provide a fuzzylogic inference system (FIS); provide an adaptive neural network; andprovide an output interface for controlling said user interface; saidFIS and adaptive neural network for receiving constraints on antecedent,scaling, and consequent parameters and input vectors, thereby definingsets of acceptable antecedent, scaling, and consequently parameters andinput vectors and for receiving initial antecedent, scaling andconsequent parameters and an initial input vector from said active inputinterface and said expert system and for for each of a series ofinstants in time: modeling a rate of change of an output vector of saiduser interface as a product of a matrix of elements and a rate of changeof an input vector of said system, elements of said matrix dependentupon antecedent, scaling, and consequent parameters and at least oneelement from an said input vector; and minimizing a norm of said matrixto obtain optimized antecedent, scaling,, and consequent parameters andan optimized input vector.
 20. The computer readable medium of claim 19wherein said FIS and adaptive neural network are also for: obtaining anestimated Jacobian matrix based on said optimized antecedent, scaling,and consequent parameters and said optimized input vector; constrainingacceptable input vectors to those having a norm equivalent to aconstant; repetitively: (a) selecting a fixed input vector meeting, saidconstraining step; (b) selecting initial antecedent, scaling, andconsequent parameters; (c) minimizing a square of a norm of said matrixto obtain a valued matrix and corresponding antecedent, scaling, andconsequent parameters; (d) provided a square of a norm a difference of(i) an identity matrix and (ii) a product of (A) said estimated Jacobianmatrix and (B) said valued matrix is less than a pre-set tolerance,storing a norm of said valued matrix and said corresponding antecedent,scaling, and consequent parameters; and selecting a stored norm of saidvalued matrix having a smallest value and selecting antecedent andconsequent parameters corresponding; with said selected stored norm astuned antecedent and consequent parameters.